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DETAILED ACTION 

1 . This Office action for U.S. Patent Application 1 0/802,428 is responsive to tine 
Appeal Brief filed 19 January 2010, in reply to the Final Rejection of 20 August 2009. 

Claims 1-3, 5, 7-15, 17, 19-25, and 27 are pending. 

2. In the previous Office action. Claims 1-3, 5, 7, 10-15, 17, 19, and 22-25 were 
rejected under 35 U.S.C. § 103(a) as obvious over "Motion-based Segmentation Using 
a Thresholded Merging Strategy on Watershed Segments" {de Smet) in view of "K- 
Harmonic Means-A Data Clustering Algorithm" (Zhang). Claims 8, 9, 20, and 21 were 
rejected under 35 U.S.C. § 103(a) as obvious over de Smet in view oi Zhang and in 
view of "A Video Segmentation Algorithm for Hierarchical Object Representations and 
its Implementation" {Herrmann). Claim 27 was rejected under 35 U.S.C. 103(a) as 
obvious over de Smet in view o1 Zhang and in view of U.S. Patent No. 6,084,912 A 
(Reitmeier). 

3. In view of the Appeal Brief filed on 1 9 January 201 0, PROSECUTION IS 
HEREBY REOPENED. New grounds of rejection under 35 U.S.C. §§ 101 and 103 are 
set forth below. 

To avoid abandonment of the application, applicant must exercise of the 
following two options: 

(1 ) file a reply under 37 § CFR 1.111 (if this Office action is non-final) or a reply 
under 37 CFR § 1 .1 13 (if this Office action is final); or 
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(2) initiate a new appeal by filing a notice of appeal under 37 CFR § 41 .31 
followed by an appeal brief under 37 CFR § 41 .37. The previously paid notice of appeal 
fee and appeal brief fee can be applied to the new appeal. If, however, the appeal fees 
set forth in 37 CFR § 41 .20 have been increased since they were previously paid, then 
applicant must pay the difference between the increased fees and the amount 
previously paid. 

A Supervisory Patent Examiner (SPE) has approved of reopening prosecution by 
signing below: 
/Mehrdad Dastouri/ 

Supervisory Patent Examiner, Art Unit 2621 

Response to Arguments 

A. Claims 1-3, 5, 7, 10-15, 17, 19, and 22-25 were rejected under 35 U.S.C. § 
103(a) as unpatentable over "Motion-based Segmentation Using a 
Thresholded Merging Strategy on Watershed Segments" (de Smef) in view 
of "K-Harmonic Means — A Data Clustering Algorithm" {Zhang). 
1. Claims 1-3, 7, 10-12,25. 

Applicant makes two arguments regarding Claim 1 : first, that the parameter q(i,k) 
in the Zhang reference is not a "probability", as asserted in the previous Office actions, 
and second, "that a person of ordinary skill in the art would not have been prompted to 
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combine the teachings of the de Smet and Zhang references. The first argument is 
persuasive and the second is not. 



a) Membership probability 

Applicant states that the parameter q(i,k) in equations 6.2 and 7 o1 Zhang has a 
dimension of distance, and so cannot be a probability, which is inherently 
dimensionless. Applicant confirms that parameter d(i,min) is a distance measurement. 
Upon further review of Zhang, it appears that the true probability function is p(i,k), which 
is shown in equation 6.4 to be dimensionless. This is also confirmed in page 7, which 
calls p(x|m) a "prior probability" and p(m(l)) a "mixing probability". Zhang is 
reinterpreted with p(i,k) mapped with the claimed membership probability. This 
substantive re-interpretation o1 Zhang is considered a new ground of rejection. This 
Office action is non-final ^ . 



b) Reason to combine 

Applicant states that due to "significant differences" between the claimed 
invention and the prior art references, one of ordinary skill in the art would not be 
motivated to combine their teachings. The examiner disagrees. 

The de Smet reference was directed to a "motion-based image segmentation" 
based on a K-means clustering method. De Smet, § 4. De Smet noted that the K- 



^ The new claim rejections under 35 U.S.C. § 101 based on revised interpretation of 
Nuijten are also new grounds of rejection that make this action non-final. 
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means method could "perform quite well", particularly since it "distinguished the areas 
with smaller or different background motion". Id. De Smef concluded by inviting future 
improvements, such as replacing components with "various other techniques", and 
exploring "other regrouping and clustering strategies" with the possibility of "even better 
results". Id. 

The Zhang reference describes the K-Harmonic Means clustering algorithm. 
Zhang, abstract. Zhang faults the K-means clustering method due to its sensitivity to 
the initialized positions of clustering centers. Id. The K-Harmonic Means method is 
specifically designed as an alternative clustering method to the K-means method, which 
improves on the K-means method by eliminating the problematic initialization sensitivity. 
Id. 

In combining two prior art references to show obviousness under 35 U.S.C. § 
103(a), "there must be some articulated reasoning with some rational underpinning to 
support the legal conclusion of obviousness". KSR Int'l Co. v. Teleflex Inc., 550 U.S. 
398, 418 (U.S. 2007) {quoting In re Kahn, 441 F.3d 977, 988 (Fed. Cir. 2006)). One of 
these rationales noted by the Supreme Court is "the application of a known technique to 
a piece of prior art ready for the improvement". Id. at 401 . To use this test as a basis 
for rejection, an examiner must 1) find "that the prior art contained a 'base' device 
(method, or product) upon which the claimed invention can be seen as an 
'improvement"'; 2) must find "that the prior art contained a known technique that is 
applicable to the base device (method, or product)"; 3) must find "that one of ordinary 
skill in the art would have recognized that applying the known technique would have 
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yielded predictable results and resulting in an improved system"; and 4) present 
"whatever additional findings based on the Graham factual inquiries may be necessary, 
in view of the facts of the case under consideration, to explain a conclusion of 
obviousness". MPEP § 2143(D). 

In this case, 1) the de Smet reference contains a "base" method of motion 
segmentation in images, using the K means clustering method. 2) The Zhang reference 
contains the known K-Harmonic Means clustering technique. Since it is designed as an 
improved K-Means technique, as presented in de Smet, it is presumed to be applicable 
wherever the K-Means technique is, such as in motion segmentation. 3) Since the K- 
Harmonic Means clustering technique solves a specific known problem with the K- 
Means clustering technique: improper sensitivity to initialization, one having ordinary 
skill in the art would recognize the predictable results and the improved system in which 
the K-Harmonic Means technique replaces the K-Means technique, namely: greater 
freedom in setting uncertain initial values (or even random or evenly-spaced values) for 
clustering centers during initialization, with less concern over producing inaccurate 
clusters. 4) It does not appear that a formal inquiry into Graham is necessary, but the 
Graham factors will be briefly discussed. Regarding the scope and contents of the prior 
art, see the above discussion of de Smet and Zhang earlier in this paragraph. 
Regarding the difference between the prior art and the claims, neither of the two prior 
art references individually disclose the claimed use of K-Harmonic Means for the 
application of motion trials, with de Smet using the old K-Means technique for this 
application and Zhang only giving a general, theoretical overview of the K-Harmonic 
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Means technique without details of applications. Regarding the level of ordinary skill in 
the art, this would be a high level of skill-higher than in most video processing patent 
applications due to the relatively abstract nature of the claims-but within the scope of 
the de Smet and Zhang references and the present application, there is no indication 
that any of these three documents is more difficult to comprehend than the other two. 
Regarding additional objective evidence. Applicant has not presented any evidence of 
secondary considerations. 

Based on the facts presented within the prior art documents, there is no evidence 
in the record that the well-established "base plus improvement" test of obviousness was 
improperly used. 

2. Claim 5. 

Applicant's assertion against claim 5 is not persuasive. In the initial iteration of 
the recursive K-Harmonic Means algorithm oi Zhang, every component of the algorithm 
is inherently initialized, including the regression function, as the result of the first 
iteration, as the values input as initialization center points cascade through the 
algorithm. 

3. Claims 13-15, 19, 22-24. 

Applicant does not argue Claim 13 and its dependent claims on their own merits, 
but merely as Claim 13 is analogous in scope with Claim 1 . Examiner does not dispute 
Applicant's assertion that the motion estimation method described in the "wherein" 
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clause in Claim 13 is analogous to the method described in Claim 1 . The examiner 
declines to discuss these claims further than what is already stated in the Final 
Rejection and the discussion of Claim 1 in § A.I . 

4. Claim 17. 

Applicant correctly asserts that Claim 17 is analogous in scope with Claim 5. The 
examiner's response to the argument against Claim 5, presented in § A.2., is fully 
incorporated as applicable to Claim 17. 

B. Claims 8, 9, 20, and 21 were rejected under 35 U.S.C. § 103(a) as 

unpatentable over de Smet in view of Zhang and further in view of "A Video 
Segmentation Algorithm for Hierarchical Object Representations and its 
Implementation {Herrmann). 
1. Claims 8, 9, 20, 21. 

Applicant does not argue Claims 8, 9, 20, and 21 on their own merits, but merely as 

dependent on their base claims. 
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C. Claim 27 was rejected under 35 U.S.C. § 103(a) as unpatentable over de 

Smet in view of Zhang and in view of U.S. Patent No. 6,084,912 [Reitmeier). 
1. Claim 27. 

Applicant argues Claim 27 is allowable both as analogous in scope with Claim 1, 
discussed above in § A.I ., and on its own merits. Applicant states that since Reitmeier 
discloses a "video coder", the examiner erred in relying on this prior art to teach a 
general purpose computer, as examiner interpreted the means-plus-functions claim 
language in view of the specification under 35 U.S.C. § 1 1 2, H 6. 

In the Final Rejection, examiner relied on col. 2: lines 64-67 o1 Reitmeier as 
disclosing the claimed general-purpose computer. This section of Reitmeier reads: "In 
fact, the video encoder can be implemented as a software application residing in the 
memory of a general purpose computer (not shown) or stored within a storage medium, 
e.g., a disk". Applicant fails to specify 1 ) how this section of Reitmeier does not or 
cannot disclose the claimed general purpose computer despite its plain meaning to the 
contrary; or 2) why the examiner erred in interpreting Claim 27 under 35 U.S.C. § 1 12, ^ 
6 as directed to a general-purpose computer, such as by presenting a preferred 
alternative interpretation in the specification. Applicant has not met its burden in 
overcoming the prima facie case of obviousness with respect to Reitmeier, and so, the 
rejection of Claim 27 is sustained, incorporating the new interpretation o1 Zhang. 



Claim Rejections - 35 USC § 101 

4. 35 U.S.C. 101 reads as follows: 
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Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

5. Claims 1-3, 5, and 7-12 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to non-statutory subject matter. Current Office policy requires that 
the phrase "non-transitory" be added to and the word "storage" be removed from all 
Beauregard claim preambles. See "Subject Matter Eligibility of Computer Readable 
Media", 1351 Off. Gaz. Pat. Office 212 (February 13, 2010). 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 1-3, 5, 7, 10-15, 17, 19, and 22-25 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over "Motion-based Segmentation Using a Thresholded Merging 
Strategy on Watershed Segments" {de Smet) in view of "K-Harmonic Means-A Data 
Clustering Algorithm" {Zhang). De Smef discloses using an iterative segment-merging 
technique to determine information for an image (abstract). 

Regarding claims 1,13, and 25, in de Smet, an initial motion field is first 
determined with a block-matching technique on 4 x 4 blocks. De Smet, § 2.1 . These 
initial block motion vectors are used for the initial segmentation. Id. at § 2.3. Then, the 
step of performing the block-based motion estimation is the claimed step of "providing 
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data points". Next, tine segments are iteratively merged according to similar or shared 
motion, according to the K-means clustering algorithm. Id. This is the claimed step of 
"clustering the data points". When this process is finished, the result is a series of large 
segments corresponding to distinct moving regions of an image, each with an 
associated motion vector. Id. This is the claimed step of "providing motion estimation". 
However, the present invention specifies performing regression clustering according to 
a K-Harmonic Means function, which is not the same as the K-means function of de 
Smet. 

Zhang discloses the K-Harmonic Means data clustering algorithm. Regarding 
claim 1 , 3, and 25, Zhang teaches selecting K centers m(l) from N data points x(i) (pg. 
1), initializing center points (pg. 2) and performing an initial iteration (pg. 5), and 
calculating distance d(i,l) between data point x(i) and center point m(l) (pg. 4), 
calculating membership probability p(i,x) based on the distance via parameters q(i,k) 
and q(i) (pg. 5), and stopping when the recursively-calculated performance value 
stabilizes, that is, when its change with each iteration becomes small (pg. 5). Then, the 
K calculations of centers m(l) are the claimed "regression functions" for performing 
regression clustering according to the K-harmonic means function. The calculation of 
each iteration of the recursive function is the claimed recalculation based on the 
membership probability, as m(k) is dependent on probability function p(i,k). The 
stabilization is the claimed "stopping criterion". 

De Smet et al. discloses the claimed invention, except for using K-Harmonic 
Means function to perform regression clustering. Zhang et al. teaches that it was known 
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to perform data clustering witli the K-Harmonic means function. Therefore, it would 
have been obvious to one having ordinary skill in the art at the time the invention was 
made to perform clustering based on a K-Harmonic means function, as taught by Zhang 
et al., rather than a linear function such as the K-means function of de Smet et al., since 
Zhang et al. states In the abstract that K-harmonic clustering is less sensitive to 
detrimental effects from sub-optimal initialization than conventional clustering 
techniques. Additionally, the K-harmonic means function was specifically designed as 
an improved version of the K-means function, which "significantly improves the quality 
of clustering results" compared with K-means (abstract). Then, the de Smet et al. 
algorithm contains a basic method which the present invention is an improvement 
thereof. The prior art Zhang et al. describes a known technique, the K-Harmonic means 
clustering algorithm, applicable to the base segmentation method of de Smet et al., by 
substituting it for the K-means algorithm. Then, one having ordinary skill in the art 
would have recognized that applying the K-harmonic means algorithm of Zhang et al. to 
de Smet et al. would have yielded the predictable result of "significantly [improved] 
quality of clustering results" and resulted in an improved system. Therefore, it is 
respectfully submitted that the use of K-harmonic clustering in de Smet et al. is 
considered obvious, since it has been held that applying a known technique to a known 
method ready for improvement to yield predictable results involves only routine skill in 
the art. Dann v. Johnston, 425 U.S. 219, 230, 189 USPQ 257, 261 (1976); In re Nillsen, 
851 F.2d 1401, 1403, 7 USPQ2d 1500, 1502 (Fed. Cir. 1988). 
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Regarding Claims 2, 3, 14, and 15, as previously mentioned, de Smet produces 
a motion vector for each segment in an image. De Smet, § 2.3. As a result, the most 
important moving areas are determined. Id. at § 3. This is the claimed production of 
motion vectors and at least one motion path. 

Regarding Claims 5 and 17, in Zhang, a clustering in which initialization is 
randomized is described (pg. 11). Since Applicant admits in page 10 of the 19 January 
Appeal Brief that this initialization is an initialization of the algorithm as a whole, it 
inherently contains an initialization of the regression function. 

Regarding Claims 10 and 12, in de Smet, pixels are set as (x, y, t) triples, with x 
and y as spatial coordinates and t as a time coordinate. De Smet, § 2.2. 

Regarding Claims 1 1 and 23, de Smet illustrates motion fields. De Smet, figs. 3- 
6. Although these motion fields are not shown as overlaid on the images, the examiner 
takes Official Notice that it was well-known in the art at the time of the invention to 
display a motion field superimposed on an image to provide a visual representation of 
motion vectors. 

Regarding Claims 12 and 24, de Smet illustrates highlighted motion segments 
overlaid on an image. De Smet, figs. 11, 12. 

8. Claims 8, 9, 20, and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over de Smet in view o1 Zhang as applied to claims 1 and 13 above, and 
further in view of "A Video Segmentation Algorithm for Hierarchical Object 
Representations and its Implementation" {Hermann). Claims 8, 9, 20, and 21 disclose 
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using color information to segment techniques, but de Smet on\y discloses "standard 
watershed techniques" to perform initial segmentation without providing details. De 
Smet, § 2.2. 

Hermann discloses a method for image segmentation to extract objects from a 
moving image. Regarding Claim 8, 9, 20, and 21, after an initial block-matching motion 
estimation, similar to that in de Smet, images are segmented according to specific color 
information, followed by shape analysis, and lastly motion analysis to merge regions to 
determine objects. Herrmann, § II. This color, shape, and motion information form the 
claimed "predetermined criteria". In color analysis, a region is determined as 
homogeneous if the pixel difference in the region is below a threshold. Homogenous, 
connected areas are determined as "quasi-flat zones". These quasi-flat zones are 
further processed and become the basis for further segmentation. Id. at § II. B. Then, 
the color analysis is the claimed step of "portioning data according to color". 

De Smet, in combination with Zhang, disclose a majority of the features of claims 
8, 9, 20, and 21 of the claimed invention, as discussed above, except for color 
segmentation. Hen-mann teaches that it was known to segment a moving image 
according to color. Therefore, it would have been obvious to one having ordinary skill in 
the art at the time the invention was made to determine image segmentation by color as 
taught by Herrmann, since Herrmann teaches in p. 205, H 3, that color analysis 
produces the most accurate type of segmentation. 
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Claim 27 is rejected under 35 U.S.C. 103(a) as being unpatentable over de Smet 
in view o1 Zhang and in view of U.S. Patent 6,084,912 A (Reitmeier). Claim 27 is in 
means-plus-function format, complying with 35 U.S.C. 1 12, H 6. Accordingly, the 
"system" of claim 27 will be limited to a general-purpose computer such as a PC, as 
illustrated in figure 5 of the specification of the present invention. Although it is implied 
that the algorithms of de Smet and Zhang are computer-operated, neither de Smet nor 
Zhang explicitly teach this. 

Re/fme/er discloses a video encoder. This encoder may operate on MPEG-4 
video (col. 1 : line 57), as specified in 6, 7 of the present invention as a codec on 
which the present invention is applied. Regarding Claim 27, the encoder of Reitmeier 
may operate as a software application on a general-purpose computer. Reitmeier, col. 
2: lines 64-67. 

De Smet, combined with Zhang, discloses the claimed invention except for 
encoding video on a general-purpose computer. Re/Yme/er teaches that it was known to 
implement an MPEG-4 encoder as software embedded on a computer. Therefore, it 
would have been obvious to one having ordinary skill in the art at the time the invention 
was made to encode the system of de Smet and Zhang in a software MPEG-4 encoder 
embedded on a computer, as taught by Reitmeier, in order to perform computationally 
complex functions such as motion compensation, quantization, and variable-length 
encoding inherent in the video coding process. 
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Conclusion 

9. Examiner formally requests an interview witli Applicant's Representative prior to 
the written reply to this Office action. Proposed topics of discussion include the support 
for the claimed regression function in the specification, Applicant's interpretation of 
equations 6.1-6.5 In the Zhang reference, and any difference between the presently- 
disclosed K-Harmonic Means function and the K-Harmonic Means function as 
presented in the Zhang reference. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David N. Werner whose telephone number is (571 )272- 
9662. The examiner can normally be reached on Monday-Friday from 8:30 to 5:00. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on (571) 272-7418. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 



Application/Control Number: 10/802,428 Page 
Art Unit: 2621 

USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



ID. N. W./ 

Examiner, Art Unit 2621 
/Mehrdad Dastouri/ 

Supervisory Patent Examiner, Art Unit 2621 



